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ABSTRACT 

Research was conducted to deternine the effectiveness 
of coBputer-managed instruction (CMI) as an aid to the nastery of 
factual content, as compared to the method of freguent, pre-announced 
quizzes^ One section of an undergraduate education course received 
conventional instruction along with quizzes; another sectio:i used a 
CMI program whose major features included nastery learning, 
self-pacing, self-instruction, provision for individual differences, 
and extensive record keeping. All other instructional activities were 
held constant across both groups, Pre-test evaluation of content 
knowledge yielded no significan differences between the groups prior 
to treatments; post-test results indicated that the CMI group 
achieved significantly greater mastery of the factual content, 
(Author/LB) 
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THE EFFECTS OF COMPUTER MAMAGEO INSTRUCTION ON 
CONTENT LEARNING OF UNDERGRADUATE STUDENTS 

OO Joan K. Honeycutt, Ed.D. 

^ Orange County Department of Education, California 

oo 

^ In the typical undergraduate course, students are required to read 

idj a textbook which provides the basic course content or subject matter 

Information. Many students, however, do not read their textbook until 
the day or two preceding the ''midterm" and again just prior to the "final." 
In terms familiar to most instructors, the students were "cramming" just 
prior to taking a test and not keeping up with their reading on a daily 
basis • Consequently, the instructor needs to devote class time to the 
presentation of information which was available in the textbook rather 
than helping students explore their attitudes toward, aj|id their under- 
standing and application of the content upon which the course is based. 

Many different methods have been utilized by instructors In order 
to facilitate periodic.mastery of background material on the part of 
students in the course; e.g., pop quizzes, weekly quizzes, summaries to 
be handed in on a frequent regular basis, etc. Computer Managed Instruction 
(CMI) is a relati.ve newcomer in the field of educational technology, and 
is one method which could be used to facilitate content mastery. 

Baker (1971) lists the four major functions of a computer managed 
instructional program as test scoring, diagnosing, prescribing and 
reporting. In his report, Daker describes several variations of the use 
^f computers in the management of instruction. These include the Instruc- 
tional flanagement System (IMS) developed by Systems Development Corporation 



us OgPA<»TWENTOFH6UTH, 

NATIONAL lk5t(TUtt OF . 
■ . ■ - EDUCATION . 

VMiS DOCUVesr has: seen RtJ*RO. 
OUCED EXACTlV AS ftECtlVED Pr6m^ 
; THE f^ERSONiOR ORGANtZATiOK ORfidlN 

; . , : " A^J^^G «t pGi^TS Of VIEW OR op^ni^ns 

Q ■ SMteO DO Ndt NECESiARltV RtPRE 

- -- . : . .. f ENtof r iciAL NATiosfAL ^KSTiture Of, 

: EDUCAtlb^ POSITION OR POUCY : 



.ERIC 



(Sllberman, 1968) » the Individually Prescribed Instruction Management and 

Information System (IPI/HIS) at Pittsburgh Learning Research and Development 

Center (Glaser, 1969), Program for Learning In Accordance with Needs 

(PLAN) developed by the American Institute for Research (Flanagan, 1969), 

Teaching Information Processing System (TIPS) developed by Kelley (1968) 

at the University of VMsconsIn, and thf; Individualized Mathematics 

Curriculum Project (IMCP) under development since 1964 at Wisconsin 

V 

(De Vault, et. al., 1969). For a comprehensive review of the use of 
computers In education, th& reader Is referred to "Information Systems 
Applications In Education" by Sllberman and Fllep (1968). 

The use of Computer-Managed Instruction (CMI) as an aid to teaching 
may be viewed as an attempt to apply a systems approach to education 
(Brudner, 1968). In this capacity CMI serves as an Information system, 
keeping track of and providing Information about each student's progress. 
In its more sophisticated use, CMI provides for diagnostic and prescriptive 
assignments based on student performance (Finch, 1969). While none of 
these functions are dependent upon the use of a computer, such use is 
the most efficient and expedient means for managi'ng an indiyidualized 
program of instruction. 

For theoretical reasons, this author selected CMI as a medium to 
facilitate mastery of content on the part of students. The purpo se of 
the study reported herein was to examine the effectiveness of CMI as 
an aid to mastery of background Information as compared to the method 
of frequent, pre-annoupced, classroom quizzes. 



BACKGROUND OF THE STUDY 

Based on the principle of mastery learning» a Computer Managed 
Instruction (CMI) program was developed which required a minimum 
performance of 90 percent correct responses for each unit of work. 
Multiple-choice questions based on Child Development and Personality 
(3rd ed.) by Mussen, Conger, and Kagan (1969) were obtained from the 
teacher's manual, and others were developed by the experimenter for 
use In the program. Each of these questions was used as a test Item 
during academic quarters preceeding the Implementation of CHI and 
subsequently subjected to Item analysis. On the basis of these analyses, 
questions were selected for the test pool which was then used In the 
CMI program. \ 

Readings from the textbook v^ere divided Into seven units which 
were further subdivided Into two or three topics per unit. The test 
pool for each unit was composed of twenty items equally divided among 
topics* Upon completion of the assigned readings for a unit, students 
were directed to an IBM 2741 communications terminal where they were 
individually administered a quiz. This quiz consisted of ten questions 
drawn at random from the tv/enty-item test pool and equally divided among/ 
between topics. 

Based on the assumption that students would learn more If they were 
not told v/hether their answers were correct or incorrect, the CMI program 
was designed to furnish students with feedback information at the 
completion of a testing session in terms of the number of correct responses 
emitted per topic. 



students whose responses met the 90 percent criterion of correct- 
ness, were instructed to proceed with tjjg next unit. Those students 
who responded with 70 - 80 percent correct responses on the quiz, were 
assigned supplemental reading(s) covering topic{s) in which difficulty 
was encountered. Upon completion of this supplemental reading assign- 
ment, students had to pass a computer-administered quiz based on these 
readings at the level of 90 percent in order to proceed to the next unit. 
If a student failed to meet this level of mastery, he was directed tx) see 
his instructor. In the event that a student failed to receive at least 
70 percent correct responses aftef* two trials on a unit quiz, he, too, 
was directed to see the instructor. Individual conferences were then 
arranged for the purpose of helping these students with any problem or 
problems which he may have had 
in understanding the material. 
Subsequently, students resumed 
activity with the next unit. 
Completion of the CHI portion 
of the course was required and 
constituted minimum performance 
for all students. 
(See Figure 1.) 
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Figure 1 



Computer Managed Instruction was Initiated during Fall quarter, 
1971. During this quarter the program was debugged and the following 
changes were made; 

1. Criterion for pass was changed from 90 percent correct to 80 
percent correct. This decision was based on the observation made during 
student interviews with the instructor that frequently students failed 
to get 90 percent correct because they had misread a question, pushed 
the wrong key on the terminal, or became flustered rather than due to 

a lack of understanding of background material. * 

2. Criterion for additional reading assignmentf was changed to 
60 - 70 percent correct in keeping with the change in 1, above. 

3. Criterion for failure of a unit was changed to 50 percent 
in keepingjyith changes in 1 and 2, above. 

During the trial quarter of CMI, it was noted that students were 
going in groups of two and three to the computer terminals and taking 
the test more or less as a "group endeavor." This "group endeavor" 
seemed to involve one student sitting at the terminal and one or more 
students seated around him--an with books and notes in hand. As 
questions were typed by the terminal, the group of students were all 
flipping through their bookfi and notes trying to determine the correct 
answer before the time limit expired. The computer terminals were not 
monitored, and there was great concern regarding student cheating at 
this time. In order to control for cheating, students were Informad 
that they would be tested on the final class meeting to determine whether 
or not they had mastered the content, themselves, or had had someone 
else take the tests for them. The results of this test would be examined 



6 

to detegjine if any of the students scored lower than the group as a 
' * **hole. If this did occur, the student{s) would be called in, individually, 

and an oral examination would be administered by the instructor. 

V • • 

METHOD 

Subjects 

Junior and senior students at the Ohio State University majoring in 
early and middle childhood education during Winter quarter, 1972, served 
as subjects for this study. Two sections of the required course in child 
development which mot at the same time but in different locations served 
as the experimental (n » 29) and control {n = 34) groups. All students 
were assigned to sections as part of the normal registration procedure at 
the institution. While subjects were not randomly assigned to treatment 
groups, their assignment was by the corr.putorized registration system, and 
I there is no reason to suspect bias between the two groups due to section 

assignments. 
Treatment 

The experimental and control groups were taught by instructors with 
essentially equal experience in teaching the course. Treatment groups 
differed in^hat students in the experimental group received Computer 
Managed Instruction (CMI), while students in the control group were given 
foiiv (4) pre-announced quizzes composed of questions selected by the 
instructor from the same test pool as the CMI program in class and a final 
essay examination. 

Lectures covering textbook material were omitted from class activities 
in both groups. Class sessions for both groups consisted of: 
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1. films 

2. discussion of films based on textbook Information 

3. implications of these films for the classroom teacher 

4. role playing 

5. presentation and application of techniques for case study 
Since the control group spent class time taking quizzes, students 

in the experimental group were released from two class sessions in order 
to equate total class time betwee?» the two treatment groups. 

Students in both treatment groups spent two hours per week 
participating in and observing children in a public school setting. 
On the basis of this participation-observation experience, students in 
both groups were required to complete an extensive case study on one of 
the children with whom they had been working. 

Final grades for the course were determined differently between 
treatment groups. Subjects in the experimental (CMI) groop^. received 
a "C" for completion .of the CMI portion only. Grades of "A" and "B" 
were determined on the basis of superior quality of case study. In 
contrast, subjects in the control {no CMI) group received letter grades 
for their performance on each of the quizzes and their final essay 
examination. Final grades for this group were detormined on the basis 
of test grades and quality of case study. 
Procedure 

During the first class meeting of the quarter, both treatment groups 
were administered a pre-test which consisted of 50 items from the course 
test pool. The same test was administered at the final class meeting as 
a post-test. Prior to taking the test, subjects in both groups were 
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Informed that research was being conducted Into the effectiveness of 
CMI and that the test results would be used for purposes of this evaluation 
study. The experimental group was further informed that the post-test 
would be utilized as a check that students were doing their own work at 
the terminals and that except in the event of very low performance on the 
post-test, the results would not effect the final grades in the course. 

Prior to administration of the post- test, students were requested 
not to study. The post-test given for purposes of this study was admin- 
istered to both the experimental and control groups at the same time: 
five days prior to the final examination which was administered to the 
control group. This was done In order to control for possible contamination 
of post-test results due to students in the control group studying for 
their final examination. No final examination was given to the experimental 
group, as their mastery of content was evaluated exclusively through CMI. 
Results of the pre-test and post-test for both*groups are presented in 
Table 1 , below. 

Table 1 

Pre-Test and PosL-Test Results 



Treatment 


Pre-Test 


Post-Test 


n 


Group 




s 


X 


s 




Experimental (CMI) 


23.3 


2.98 


39.9 


4.86 


29 


Control (no CMI) 


21.9 


4.56 


34.3 


4.93 


34 



§ 
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RESULTS 

\ 

X Pre-test scores for the experimental and control groups were 
compared In order to determine whether or not the two groups had 
equal background knowledge prior to treatments. The ^^^^ statistic 
was employed to test for homogeneity of variance with a finding of 
no significant difference between variances. A t-test was conducted 
on pre-test scores which yielded a t value of 1.327 indicating no 
significant difference between the groups in the amount of factual 
knowledge prior to treatments. 

Post-test scores were analyzed with a t-test which yielded a 
t value of 4.455 {p< .005) favoring thp experimental group. These 
results are summarized in Table 2, below. 

^able 2 

Results of t-test comparison of 
performance between experimental and control groups 

as 





Experimental 


Control 


t 




■% 


s 


X 


s 




Pre-test 


23.3 


2.98 


21.9 


4.56 


1 .327 n.c. 


Post-test 


39.9 


4.86 


34.3 


4.93 


4.455* 



*p< ,005 
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SUMMARY AND DISCUSSION V. 

In order to facilitate mastery of content material, meet Individual 
student needs and provide for student di f I'erences , a Computer Managed 
Instruction (CMI) program was developed by the author which had the 
following features: (1) Self-pacing . Course of study was divided into 
seven units, and within thft time frame of one academic quarter, students 
were free to complete the units at their own rate, (2) Mastery Learning . 
A minimum performance of 80 percent correct responses was required for 
each unit of work, (3) Individual d i fferences . Materials were developed 
for each unit which served as supplenrsr.tary readings for students who were 
not able to meet the 80 percent mastery level. For the few students 
who continued to experience difficulty, individual conferences were 
scheduled with the instructor, (4) Self-teaching . Students Vere not 
told which of their answers were correct or incorrect, but were furnished 
feedback at the completion of each session in terms of the number of 
correct responses emitted per topic, (5) Re cord keeping . Summaries of each 
student's status were available to instructor at any time through the 
computer terminal . 

The study reported herein was designed to examine the effectiveness 
of CMI as an aide to mastery of background information as comi.ared to 
the method of frequent, pre-announced classroom quizzes. 

The results of this study indicated a highly significant difference 
in post-test standing (p< .005) between the experimental (CMI) and control 
(no CMI) groups v/hich favored the experimental group. It would appear 
that Computer Managed Instruction (CMI) based on mastery of content and 
self-pacing is a highly effective aid far maximizing content learning by 
undergraduate college students, 

o ' ' \ ■ ' 
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It should be pointed out that six subjects In the experimental 
group (21^) had not completed a^' seven units prior to post-testing. 
Conceivably, these subjects had not yet read all the material covered 
by the post-test. Nevertheless, the mean post-test scoro for tho 
experimental group was significantly higher than that of the control 
group. 

Of Interest was the observation that the experimental group mean 
on the post-test was 39.9 or 80 percent of the questions correct--the 
same criterion as that required for each unit of the CMI program. 
This could be interpreted as a chance factor*, however, it could also 
be hypothesized that since the experimental subjects had been required 
to attain a minimum response rate of 80 percent correct, they were, 
in fact, still operating under this condition. 

The "group endeavor" described in the Background of the Study 
which consisted of students working together at the terminals was an 
unanticipated bonus. Observation of this group learning was a 
beautiful and exciting teaching-learning situation to witness. We In 
the field of education, talk about the merit found in group learning 
vs. individual learning situations, and the results of this study 
would certainly lend support to group learning. 

Rigid control was not maintained as to the exact content of what 
happened in each class session. While the two instructors had 
comparable experience In teaching the course, used the same materials, 
and the same course objectives, the actual effect of individual 
differences between instructors cannot be ascertained. Since the 
material covered in class was of the nature of expansion and application 



of textbook material rather than direct presentation of this material^ 
It would seem reasonable to assume that any bias relating to instruc- 
tor differences would not be reflected In the criterion measures used 
In this study. 

As part of a university-wide project, students In the experimental 
group were administered an attltudinal questionnaire during the last 
week of class. These questionnaires were answered anonymously and 
requested opinions of and attitudes toward Computer Assisted Instruction 
(CAI) and Computer Managed Instruction (CMI). The results of this 
survey indicated much frustration on the part of students with CMI. 
Problems which they identified were such things as glare on the glass 
coverings of communications terminals, difficulty with paper feed on 
the typewriter rolls, amount of noise in the rooms where the terminals 
were located, and the threat which some students felt in taking a 
test under these conditions. One would expect that such negative 
reactions would adversely affect the results for the experimental 
group. However, analysis of the data indicated that the group on CMI 
(experimental) achieved a significantly greater mastery of content 
than those not iBn CHI (control ) . One might say that the students may - 
not have liked CMI, but they definitely seemed to learn more on it. 

There is a need for replication of this study i in particular, 
one which would Include control for differences between instructors. 
For example, if two instructors were Involved in the study, each 
could teach one class with CMI and one class without CMI. 
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In no way should the results of this study be interpreted as 
offering a panacea. J(or all educational ills. The Computer Managed 
Instruction program ^ied in this study was designed for a specific 
population of students; namely, undergraduate students taking a 
required course which necessitated mastery of content background 
Information. In a course where mastery of content Is a necessary 
condition for meeting the overall objectives of the course, a 
Computer Managed Instructional program such as this would seem to 
be a very effective method to be utilized. One would suspect that 
there are many courses which have such a requirement. 
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